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CHAPTER  1: 


INTRODUCTION 

1.1  Background 

The  1988  Physicians'  Practice  Costs  and  Income  Survey  (PPCIS)  is  the  most  recent  of 
a  series  of  physician  surveys  conducted  for  the  Health  Care  Financing  Administration 
(HCFA).  One  of  its  main  purposes  was  to  obtain  data  on  the  costs  of  medical  practice. 
Such  data  have  been  used  to  support  policy-relevant  analyses  of  the  composition  of  practice 
costs  and  geographic  differences  in  the  costs  of  practice  (Thalji  et  al.,  1991).  Data  needs  are 
likely  to  increase  in  the  future  as  the  new  Medicare  Fee  Schedule  is  refined  and  as  the  health 
care  system  is  reformed.  Evaluations  and  assessments  of  previous  survey  efforts  are 
important  sources  of  information  that  can  be  used  to  help  design  data  collection  strategies  to 
meet  future  needs. 

This  report  complements  an  earlier  report  prepared  by  the  Project  HOPE  Center  for 
Health  Affairs  (Project  HOPE,  1991).  A  goal  of  this  project  has  been  to  provide  HCFA  with 
information  specific  to  the  1988  version  of  the  PPCIS.  This  information  may  be  helpful  in 
HCFA's  deliberations  on  whether  a  survey  effort  of  the  type  represented  by  the  PPCIS 
should  be  administered  by  HCFA  in  the  future  and  whether  design  changes  should  be  made, 
or  whether  other  data  collection  strategies  should  be  explored  to  meet  HCFA's  needs. 
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1.2  Overview 


In  the  PPCIS,  practice  cost  data  were  provided  by  surveyed  physicians  or  ~  for 
approximately  one-half  of  all  surveyed  practices  —  by  a  proxy  respondent,  such  as  an  office 
or  business  manager.  An  interesting  finding  documented  in  the  earlier  report  was  a  PPCIS 
"design  effect"  associated  with  respondent  type.  Total  practice  costs  reported  by  proxy 
respondents  were  about  60  percent  less  than  costs  reported  by  physicians  themselves.  This 
effect  was  observed  using  an  empirical  specification  that  has  traditionally  been  in  cost 
analyses.  The  specification  "controls"  for  a  number  of  other  factors  that  have  been  shown  to 
affect  the  costs  of  providing  physician's  services. 

One  of  the  specific  purposes  of  this  report  is  further  examine  the  design  effect 
because  its  impact  is  large  and  because  proxy  respondents  are  often  used  as  sources  of  cost 
data  in  physician's  surveys  as  a  means  of  increasing  response  rates.  Specifically,  Chapters  2 
and  3  address  the  following  questions:  For  what  type(s)  of  practices  were  cost  data  provided 
by  the  physician  versus  a  proxy  respondent?  Is  there  any  evidence  that  physicians  or  proxy 
respondents  are  more  knowledgeable  about  practice  costs?  Results  of  this  analysis  indicate 
that  there  are  systematic  differences  in  practices  for  which  physicians  and  proxy  respondents 
provided  practice  cost  data.  Furthermore,  evidence  from  the  1988  PPCIS  suggests  that 
proxy  respondents  in  larger  practices  are  more  knowledgeable  about  practice  costs  than 
physician  respondents. 

In  the  final  chapter  of  the  report,  sources  of  survey  error  that  are  applicable  to 
practice  cost  surveys  are  reviewed.  Survey  participation  and  item  nonresponse  rates  of  the 
1988  PPCIS  are  compared  with  those  of  similar  public  and  private  physician  surveys. 
Evidence  of  the  occurrence  of  logical  inconsistencies  in  the  1988  PPCIS  data  is  also 
reviewed.  The  chapter  concludes  by  questioning  whether  national  surveys  are  the  best 
vehicles  for  obtaining  practice  cost  information.  Some  alternative  data  collection  strategies 
are  discussed. 
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CHAPTER  2: 

FACTORS  RELATED  TO  USE  OF  A  PROXY  RESPONDENT 


2.1  Overview 

In  the  1988  PPCIS,  data  on  each  practice  were  provided  by  the  physician  or  by  the 
physician  and  proxy  respondents.1  Survey  design  permitted  proxy  respondents  to  answer 
certain  questions  —  including  those  on  practice  costs  —  that  could  be  answered  using  practice 
records.  Use  of  proxy  respondents  was  permitted  to  help  alleviate  the  burden  of  survey 
participation  on  physicians  (Thalji  et  al.,  1991).  This  flexibility  is  important  in  obtaining 
maximum  practice  participation  -  especially  in  surveys  such  as  the  PPCIS  that  are  used  to 
affect  physician  payment  and  other  public  policies.  At  the  same  time,  it  is  important  to 
remember  that  survey  design  features  sometimes  involve  tradeoffs.  On  the  one  hand,  use  of 
proxy  respondents  can  improve  response  rates:  on  the  other  hand,  survey  methodologists 
have  long  recognized  that  their  use  may  introduce  biases  if  the  quality  or  validity  of  the  data 
varies  by  respondent  type. 

As  noted  in  Chapter  1 ,  evidence  from  a  multivariate  analysis  of  practice  costs 
indicates  that  costs  differ  between  practices  for  which  physicians  responded  to  the  PPCIS  and 
practices  for  which  proxy  respondents  provided  answers  to  the  cost  questions.  The  purpose 
of  the  work  presented  in  this  chapter  is  to  characterize  practices  by  type  of  respondent.  Such 
information  will  be  of  help  in  trying  to  understand  the  source  of  the  design  effect. 
Furthermore,  if  practices  exhibit  systematic  variations  by  type  of  respondent,  then  if  may  be 


'Survey  data  were  provided  entirely  by  proxy  for  50  practices  -  less  than  1.5  percent  of  respondents. 
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possible  to  implement  special  survey  design  features  to  ensure  that  data  of  comparable  quality 
are  obtained  from  each  type  of  practice. 

The  next  section  describes  the  conceptual  framework  that  underlies  the  empirical 
specifications  that  are  presented  in  Section  2.3.  Findings  based  on  a  multivariate  analysis  of 
PPCIS  data  are  presented  in  Section  2.4.  The  chapter  concludes  with  a  discussion  of 
implications. 

2.2  Framework 

The  physician's  decision  to  participate  in  a  survey  such  as  the  PPCIS  can  be  analyzed 
within  the  framework  of  the  economic  theories  of  the  firm  and  production.  Marder  and 
Thran  (1989)  model  the  physician's  participation  in  the  American  Medical  Association's 
(AMA's)  Socioeconomic  Monitoring  System  (SMS)  surveys.  They  assume  that  physicians 
seek  to  minimize  the  cost  of  practice.  The  economic  value  or  return  to  the  physician's  time  - 
-  measured  by  the  physician's  wage  —  is  endogenous  in  their  model.  Survey  participation  is 
one  of  several  activities  on  which  the  physician  may  spend  his  or  her  time.  In  this  analysis, 
a  simpler,  two-stage  decision  model  is  assumed.  It  is  assumed  that  the  physician  behaves  as 
if  he  or  she  were  maximizing  satisfaction  associated  with  income  and  leisure.  Hours  of  work 
is  determined  in  the  first  stage.  In  the  second  stage,  the  physician  decides  how  to  allocate 
working  hours  and  other  practice  inputs  across  the  various  activities  of  the  practice. 

Practice  activities  include  direct  patient  care  (including  care  provided  in  the  office  as 
well  as  in  alternative  settings  such  as  the  nursing  home  and  hospital);  administrative 
activities;  and  other  activities  such  as  continuing  education.  Under  the  standard  assumption 
that  the  physician  maximizes  profit  (and  minimizes  the  cost  of  production),  resources  are 
used  in  production  so  long  as  the  contribution  of  the  "last"  unit  of  each  resource  exceeds  its 
marginal  cost  to  the  practice  —  in  economic  jargon,  as  long  as  its  marginal  revenue  product 
exceeds  its  marginal  cost.  Thus,  the  physician's  time  is  allocated  among  activities  so  that  the 
marginal  revenue  product  in  each  activity  equals  the  marginal  cost  of  the  physician's  time  in 
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that  activity.  Within  this  framework,  whether  and  to  what  extent  the  physician  engages  in  an 
activity  depends  on  the  determinants  of  the  demand  for  physicians'  services  and  the 
determinants  of  the  cost  of  production.2 

Empirical  specifications  studied  in  this  analysis  differ  from  the  Marder-Thran 
specifications.  A  regressor  in  the  Marder  and  Thran  regression  model  of  proxy  response  is 
the  predicted  value  of  the  physician's  wage,  based  on  a  model  of  the  determination  of  the 
value  of  the  physician's  time.  Instead,  empirical  specifications  used  in  this  study  include 
reported  hours  worked  by  the  physician  (assumed  to  be  determined  prior  to  the  decision  of 
whether  to  participate  in  the  survey).  The  empirical  specifications  below  also  distinguish 
between  group  practices  of  different  sizes,  whether  groups  are  single  or  multispecialty,  and 
whether  the  practice  is  incorporated  -  factors  not  examined  in  detail  by  Marder  and  Thran. 

2.3  Methods 

2.3.1  Data 

A  multiple  regression  model  was  specified  to  identify  factors  that  affect  the 
physician's  designation  of  a  proxy  respondent  during  fielding  of  the  1988  PPCIS.  Data 
include  variables  collected  directly  from  the  survey  respondent  about  the  physician  and  the 
practice  and  variables  describing  the  field  experience  compiled  by  survey  personnel  during 
the  field  period. 

2.3.2  Model  and  Hypotheses 

The  dependent  variable  of  the  model  is  whether  the  physician  delegated  survey 
responsibilities  to  a  proxy  respondent.  The  theoretical  framework  can  be  used  to  help 


:  Marginal  revenue  product  is  derived  from  the  demand  relationship  between  price  and  quantity.  In  the  same 
way,  determinants  of  physician  costs  determine  the  marginal  cost  of  services. 
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identify  factors  that  potentially  affect  the  physician's  decision  to  delegate  responsibility  for 
the  survey.  Determinants  of  the  "delegate  decision"  include  determinants  of  demand  for 
physician's  services  and  determinants  of  the  supply  of  services. 

Demand  Factors.  There  are  numerous  office  activities  associated  with  the  production 
of  physicians'  services.  Physicians  are  directly  involved  in  patient  care,  and  determine  the 
extent  of  their  involvement  and  delegation  of  tasks  to  other  care  providers  including 
physicians'  assistants,  nurses,  and  technicians.  Other  office  activities  that  must  be  staffed 
include  various  administrative  and  clerical  functions  that  are  complementary  to  service 
provision.  The  extent  to  which  physicians  allocate  their  own  time  to  each  of  these  activities 
depends  on  the  value  of  the  marginal  unit  of  time  in  that  activity,  or  how  much  the  marginal 
unit  of  time  affects  the  physician's  marginal  revenue  product.  Although  certain  types  of 
personnel,  such  as  physicians'  assistants  and  nurses,  can  substitute  directly  for  the  physician 
in  certain  patient  care  activities,  the  value  of  the  physician's  time  is  greatest  in  patient  care 
activities.  In  contrast,  other  types  of  personnel  may  have  a  comparative  advantage  in  other 
activities,  such  as  management  of  the  practice's  financial  affairs  (Scheffler,  1975). 

The  physician's  opportunity  cost  of  completing  the  survey  depends  on  the  physician's 
workload.  A  very  busy  physician  —  for  example,  a  physician  who  works  longer  hours  than 
average,  with  a  heavy  patient  load  or  many  administrative  responsibilities  -  would  be 
expected  to  rely  on  a  proxy  respondent  more  often  that  a  less  busy  physician  if  the  proxy 
respondent  has  a  comparative  advantage  with  financial  dealings.  The  practice  cost  survey 
questions  are  likely  to  be  more  burdensome  than  other  parts  of  the  PPCIS  questionnaire. 
Provision  of  accurate  answers  may  require  checking  closely  with  the  practice's  financial 
records.  In  the  models  reported  below,  "busyness"  is  measured  by  the  number  of  hours  the 
physician  worked  during  the  preceding  week  and  the  number  of  hours  squared  (HOURS  and 
HOURS_SQD  in  the  tables  below).    Note  that  hours  worked  is  included  as  an  independent 
variable  because  their  value  is  determined  in  the  first  stage  of  the  two-stage  model.  In  the 
Marder-Thran  analysis,  busyness  is  measured  by  the  predicted  value  of  the  physician's  wage. 
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Other  variables  that  are  included  in  the  model  to  account  for  differences  in  the  levels 
of  demand  facing  individual  physicians  are  whether  the  physician  is  board-certified,  physician 
specialty,  and  geographic  location.  Evidence  indicates  that  board  certification  may  increase 
demand  for  patient  care  services  (e.g.  Wilensky  and  Rossiter,  1983;  Marder  and  Thran, 
1989),  so  a  positive  relationship  between  the  categorical  variable  indicating  certification 
(BRD  CERT)  and  delegation  to  a  proxy  respondent  is  expected  a  priori.  Physician  specialty 
is  measured  using  a  set  of  the  re-coded  specialty  indicators  from  the  PPCIS  public  use  tape 
(defined  according  to  American  Medical  Association  conventions).  Three  specialty 
categories  are  included  in  the  regression  models  to  control  for  effects  relative  to  surgical 
specialties:  general  and  family  practice  (GEN),  medical  specialties  (MED),  and  "other" 
specialties  (OSPEC). 

Two  sets  of  categorical  variables  indicate  the  physician's  geographic  location:  Federal 
Census  region  of  the  practice,  and  degree  of  urbanicity  of  the  surrounding  area.  The  Census 
region  variables  indicate  whether  the  practice  is  located  in  the  northeast  Census  region 
(NEAST),  midwest  (MIDWEST),  or  south  (SOUTH),  compared  to  the  west.  Categorical 
variables  are  used  to  indicate  whether  the  practice  was  located  in  a  large  Standard 
Metropolitan  Statistical  Area  (SMSA)  (LG  SMSA)  or  rural  area  (NO_SMSA),  compared  to 
smaller  SMSAs.  Other  market  variables  that  measure  demand  determinants,  such  as  per 
capita  income  and  the  number  of  physicians  per  capita,  cannot  be  included  in  the  model 
because  the  PPCIS  does  not  identify  specific  practice  locations  within  Census  regions. 

Supply  Factors.  Practice  size  is  expected  to  directly  affect  the  physician's  decision 
to  delegate  responsibility.  Practice  size  is  directly  associated  with  practice  complexity. 
Review  of  financial  records  becomes  more  burdensome  and  cost-accounting  becomes  more 
complex  in  larger  practices.  An  individual  physician  respondent  is  less  likely  to  have  a 
working  familiarity  with  total  practice  costs  as  practice  size  increases.  Larger  practices  may 
also  experience  scale  economies.  Such  practices  can  more  fully  employ  clerical  personnel, 
such  as  bookkeepers  and  business  managers,  who  may  have  more  knowledge  of  day-to-day 
financial  affairs  than  individual  physicians. 
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The  empirical  specification  of  the  proxy  decision  and  the  equation  used  to  predict  the 
physician's  wage  used  by  Marder  and  Thran  contained  a  dummy  variable  to  distinguish 
between  solo  and  group  practices.  In  this  analysis,  practice  size  is  measured  using  a  set  of 
dummy  variables  to  distinguish  between  solo  practices  and  groups  of  different  sizes. 
Categories  have  been  constructed  so  that  regression  coefficients  can  be  used  to  compare 
larger  practices  to  the  average  solo  practice.  Categories  are  practices  with  the  number  of 
full-time  equivalent  physicians  ranging  from:  one  to  three  (DOCSl_3);  three  to  five 
(DOCS3_5);  five  to  eight  (DOCS5_8);  eight  to  20  (DOCS8_20);  and  more  than  20 
(DOCS21). 

Two  additional  practice  characteristics,  included  in  the  model  as  measures  of  practice 
complexity,  are  whether  the  practice  is  a  multi-specialty  practice,  and  whether  the  practice  is 
incorporated.  It  is  hypothesized  that  physicians  in  multi-specialty  practices  (M_SPEC  =  1, 
versus  0  otherwise)  are  more  inclined  to  delegate  survey  responsibilities  because  they  are  less 
likely  to  be  familiar  with  practice  costs  of  colleagues  of  different  specialties.  It  is  not 
obvious,  a  priori,  whether  physicians  in  incorporated  practices  (CORP=l,  versus  0 
otherwise)  are  more  or  less  inclined  to  serve  as  survey  respondents  than  solo  or  partnership 
physicians. 

Field  Variables.  Finally,  three  variables  that  describe  PPCIS  field  work  by  data 
collection  personnel  were  included  in  the  model  to  control  for  differences  in  field  operations 
across  practices:  how  late  into  the  field  period  data  collection  at  the  practice  began 
(HOW_LATE,  measured  in  days  from  the  beginning  of  the  field  period),  how  much  time  the 
data  collection  process  required  (HOW_LONG,  also  measured  in  days),  and  whether  data 
were  obtained  using  computer-assisted  telephone  interviewing  (CATI)  procedures  rather  than 
paper-and-pencil  recording  of  responses  with  subsequent  computerized  data  entry  (CADE).3 


3  Under  CATI  —  Computer  Assisted  Telephone  Interviews  —  data  are  entered  directly  into  a  computer  with 
built-in  editing  capabilities.  Approximately  51  percent  of  interviews  were  completed  using  CATI.  Computer 
Assisted  Data  Entry,  or  CADE,  was  used  in  the  remainder  of  interviews.  Interviewers  recorded  responses  using 
paper  and  pencil.  Data  were  entered  into  a  computer  at  a  later  time. 
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It  was  hypothesized  that  lateness  is  directly  associated  with  the  likelihood  that  a  proxy  will  be 
used  because  of  pressures  to  complete  data  collection.  The  length  of  the  data  collection 
period,  controlling  for  start  date,  should  be  directly  related  to  use  of  a  proxy  respondent 
because  reluctant  physician  participants  would  be  more  inclined  to  participate  by  proxy  than 
more  willing  participants.  A  categorical  variable  for  the  use  of  CATI  versus  CADE 
procedures  is  included  in  the  model  because  a  CATI  design  effect  was  observed  in  Project 
HOPE'S  earlier  analysis  (Project  HOPE,  1991). 

2.4  Findings 

Two  types  of  empirical  evidence  are  used  to  examine  issues  related  to  the  physician's 
selection  of  respondent.  First,  some  univariate  statistics  from  the  PPCIS  data  are  used  to 
compare  practices  of  physician  and  proxy  respondents.  Second,  estimates  of  the  multivariate 
model  are  used  to  identify  factors  related  to  use  of  a  proxy  respondent. 

2.4.1    Descriptive  Statistics 

The  PPCIS  data  were  used  to  profile  practices  of  physician  respondents  and  practices 
for  which  a  proxy  respondent  provided  PPCIS  data  (Table  2-1).  Results  suggest  that  busier 
physicians  were  somewhat  more  inclined  to  delegate  practice  cost  questions  to  a  proxy 
respondent.  The  average  length  of  the  preceding  work  week  was  about  2.5  hours  less  for  the 
physician  respondent  than  for  the  physician  who  delegated;  55  percent  of  physicians  who 
were  represented  by  a  proxy  worked  at  least  60  hours  during  the  week  prior  to  the  survey, 
versus  50  percent  of  physician  respondents. 

The  geographic  location  of  practices  for  which  respondents  were  used  differs  from  the 
locations  of  physicians  who  completed  the  cost  questions  themselves.  Relatively  fewer 
practices  represented  by  a  proxy  were  located  in  the  largest  urban  areas  (Standard 
Metropolitan  Statistical  Areas,  or  SMSAs).  Only  13  percent  of  physician-respondent 
practices  were  located  in  non-SMSAs,  versus  17  percent  of  practices  using  a  proxy.  Fifty- 
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Table  2-1 :  Practice  Characteristics  by  Respondent  Type 


Characteristic 

Physician's  work  hours, 
preceeding  week: 

Hours 


Fraction  with 
60  or  more  hours 


Mean* 
(Standard  Error) 
Proxy  Physician 
Respondent  Respondent 


59.91 
(0.45) 

0.55 
(0.01) 


57.41 
(0.37) 

0.50 
(0.01) 


Ratio 


1.04 


1.10 


Practice  Location: 
Large  SMSA 
Small  SMSA 
Non  SMSA 


0.28 
(0.01) 
0.55 
(0.01) 
0.17 
(0.01) 


0.40 
(0.01) 
0.47 
(0.01) 
0.13 
(0.01) 


0.70 
1.17 
1.31 


Practice  Size: 


1  physician 

1-3  physicians 

3-5  physicians 

5-8  physicians 

8-20  physicians 

More  than  20 
physicians 


0.40 
(0.01) 

0.24 
(0.01) 

0.13 
(0.01) 

0.07 
(0.01) 

0.10 
(0.01) 

0.07 
(0.01) 


0.64 
(0.01) 

0.18 
(0.01) 

0.08 
(0.01) 

0.04 
(0.01) 

0.03 

c 

0.02 
c 


0.67 
1.33 
1.63 
b 

3.33 
3.50 
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Table  2-1  (Cont'd):  Practice  Characteristics  by  Respondent  Type 


Characteristic 

Physician  is  in  a 
multispecialty  practice 


Mean* 
(Standard  Error) 
Proxy  Physician 
Respondent  Respondent 


0.17 
(0.01) 


0.07 
(0.01) 


Ratio 

2.43 


Practice  is  incorporated 


0.64 
(0.01) 


0.49 
(0.01) 


1.31 


Source:  Analysis  of  1988  PPCIS  data  by  Project  HOPE. 

Notes:  'Means  and  standard  errors  are  weighted.  The  ratio  is  not  reported  when  the  difference  between  proxy 
and  physician  means  is  not  statistically  significant  at  an  alpha  level  of  0.01.  "Value  less  than  0.005. 
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one  percent  of  physicians  in  non-metropolitan  areas  delegated  survey  responsibilities, 
whereas  only  36  percent  in  the  largest  metropolitan  areas  designated  a  proxy  respondent  (not 
displayed  in  Table  2-1).  Historically,  physicians  in  non-metropolitan  areas  have  worked 
longer  hours  and  more  weeks  per  year  (American  Medical  Association,  1990).  These 
findings  are  consistent  with  the  relationship  between  use  of  a  proxy  respondent  and  hours  of 
work  displayed  in  Table  2-1. 

Practice  size  and  complexity  are  also  related  to  use  of  a  proxy  respondent.  Solo 
practitioners  comprise  64  percent  of  practices  for  which  the  physician  responds,  and  only  40 
percent  of  practices  for  which  proxy  respondents  were  used.  Two-thirds  of  solo  practitioners 
answered  the  practice  cost  questions  themselves  (not  displayed  in  Table  2-1).  By  contrast, 
17  percent  of  proxy  respondents  are  in  groups  of  eight  or  more  doctors,  whereas  only  five 
percent  of  physician-respondents  are  in  practices  this  large.  Seventy-six  percent  of  all 
physicians  in  groups  of  eight  or  more  delegated  survey  responsibility. 

Proxy  respondents  are  more  likely  to  be  used  by  physicians  in  multi-specialty  groups. 
Seventeen  percent  of  proxy  respondents  are  in  multi-specialty  groups,  versus  seven  percent  of 
practices  with  physician  respondents.  Relatively  more  of  unincorporated  practices  were 
represented  by  physicians  than  by  proxy  respondents. 

Although  there  was  no  statistically  significant  difference  between  the  start  of  data 
collection  in  practices  that  differed  with  respect  to  type  of  respondent,  respondent  type  is 
related  to  the  period  of  time  required  to  complete  data  collection  and  data  collection  mode 
(Table  2-2).  The  survey  was  in  the  field  longer  for  practices  for  which  a  proxy  respondent 
was  used  than  when  a  physician  provided  the  practice  cost  information.  It  is  quite  likely  that 
proxy  respondents  were  used  in  some  instances  because  physicians  had  not  provided  data  in  a 
timely  manner.  Data  collection  using  CATI  was  also  more  common  in  practices  for  which 
physicians  answered  the  cost  questions. 
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Table  2-2:  Data  Collection  Timing  and  Mode  by  Respondent  Type 


Mean' 
(Standard  Error) 


Proxy 
Respondent 


Physician 
Respondent 


Ratio 


Timing  of  Data  Collection: 


Days  between  start 
of  data  collection 
and  1/1/89 


292.01 
(2.20) 


287.14 
(1.85) 


Days  between  completion 

of  data  collection  333.50 

and  1/1/89  (2.17) 


316.69 
(1.90) 


1.05 


Data  Collection  Mode: 

Computer-assisted  0.44  0.58  0.76 

telephone  interview  (0.01)  (0.01) 

(CATI) 


Source:  Analysis  of  1988  PPCIS  data  by  Project  HOPE. 

Notes:  "Means  and  standard  errors  are  weighted.  The  difference  between  the  proxy  and  physician  means  is 
not  statistically  significant  at  an  alpha  level  of  0.01. 


2.4.2  Multivariate  Estimates 

The  univariate  statistics  displayed  in  Table  2-1  is  consistent  with  the  key  hypotheses 
of  the  model  that  busy  physicians  and  physicians  in  larger  practices  are  more  likely  to 
delegate  practice  cost  questions  to  proxy  respondents.  Nevertheless,  relationships  between 
demand  and  supply  factors  and  type  of  respondent  should  be  examined  after  controlling  for 
other  physician  and  practice  characteristics.  Descriptive  statistics  for  the  sample  physicians 
included  in  the  multivariate  model  are  presented  in  Table  2-3;  estimation  results  are 
presented  in  Table  2-4. 

Three  versions  of  a  linear  probability  model  were  estimated  using  ordinary  least 
squares.4  The  first  specification  (Model  1)  includes  all  independent  variables.  The  second 
and  third  specifications  are  presented  to  highlight  the  relative  contributions  of  the  busyness, 
practice  characteristics,  and  field  variables.  In  the  second  specification  (Model  2),  field 
variables  are  deleted;  field  and  practice  size  variables  are  deleted  in  the  third  specification 
(Model  3). 

Regression  results  indicate  that  practice  size  and  complexity  account  for  much  of  the 
variation  in  respondent  type.  The  simplest  equation  —  Model  3,  including  only  the  busyness 
and  other  physician  characteristics  variables  -  accounts  for  only  4  percent  of  the  variation  in 
respondent  type  (the  R2  value,  adjusted  for  the  number  of  independent  variables,  is  0.04). 
Introduction  of  the  practice  size  and  complexity  variables  increases  the  explained  variation  by 
eight  percentage  points  (Model  2),  while  the  field  variables  only  contribute  an  additional  one 
percent  to  the  model's  explanatory  power  (Model  3). 

Geographic  location  and  practice  size  are  the  most  important  determinants  of 
respondent  type.  Furthermore,  impacts  of  these  variables  are  similar  across  the  three 


4  The  large  sample  size,  and  the  fact  that  roughly  half  of  the  sample  were  proxy  respondent  practices,  mitigate 
problems  associated  with  the  linear  probability  model  relative  to  probit  or  logit  models. 
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Table  2-3:    Variables  Used  in  Regression  Analysis  of  Whether  a  Proxy  Respondent 
Provided  PPCIS  Cost  Data 


Independent 
Variable 


Description 


Percent  With 
Characteristics 
or  Mean 


Location  of  practice: 

NEAST 

MIDWEST 

SOUTH 

WEST 


Northeast  region 
Midwest  region 
South  region 
West  region 


0.23 
0.21 
0.33 
0.23 


LG_SMSA 
SM_SMSA 
NO  SMSA 


Large  SMSA 
Small  SMSA 
Non  SMSA 


0.35 
0.50 
0.15 


Physician's  specialty: 

GEN 

MED 

SURG 

OSPEC 

BRD_CERT 

HOURS 

HOURS  SQD 


General  practice  0.15 

Medical  specialty  0.30 

Surgical  specialty  0.34 

Other  specialty  0.21 

Physician  is  board-certified  0.74 

Physician's  work  hours, 

preceding  week  58.55 

Hours,  squared  3657.24 


Number  of  full-time  equivalent  physicians  in  practice: 

SOLO 

1  physician 

0.55 

D0CS1  3 

1-3  physicians 

0.20 

D0CS3  5 

3-5  physicians 

0.10 

D0CS5  8 

5-8  physicians 

0.05 

D0CS8  20 

8-20  physicians 

0.06 

D0CS22 

More  than  20  physicians 

0.04 

M_SPEC 

Practice  is  a  multispecialty 

group 

0.12 

CORP 

Practice  is  incorporated 

0.55 
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Table  2-3  (Cont'd):  Variables  Used  in  Regression  Analysis  of  Whether  a  Proxy 
Respondent  Provided  PPCIS  Cost  Data 


Percent  With 

Independent  Characteristics 
Variable  Description  or  Mean 


HOW_LATE  Days  between  start  of  data 

collection  and  1/1/89  285.03 

HOWJ-ONG  The  number  of  days  required  to 

complete  data  collection  39.68 

CATI                                 Data  collection  by  computer- 
assisted  telephone  interview  0.53 


Source:  Analysis  of  1988  PPCIS  data  by  Project  HOPE. 
Note:  Means  and  percents  are  weighted. 
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Table  2-4:    Regression  Equations  Examining  Whether  a  Proxy  Respondent 
Provided  PPCIS  Cost  Data 


Independent 

Variable  Model  1  Model  2  Model  3 

NEAST  -0.045  -0.038  -0.063" 

(0.026)  (0.026)  (0.027) 

MIDWEST  0.032  0.030  0.050 

(0.026)  (0.027)  (0.028) 

SOUTH  0.028  0.030  0.018 

(0.024)  (0.024)  (0.025) 

GEN  -0.016  -0.011  -0.044 

(0.028)  (0.028)  (0.029) 

MED  -0.039  -0.039  -0.032 

(0.022)  (0.022)  (0.023) 

OSPEC  -0.214"  -0.225"  -0.192" 

(0.025)  (0.025)  (0.026) 

BRDCERT  -0.004  -0.003  0.056" 

(0.021)  (0.021)  (0.021) 

LGSMSA  -0.061"  -0.058"  -0.095" 

(0.020)  (0.020)  (0.021) 

NO_SMSA  0.027  0.022  0.003 

(0.026)  (0.026)  (0.027) 

HOURS  -0.005"  -0.005"  -0.003 

(0.002)  (0.002)  (0.003) 

HOURS_SQD  a"  a"  a 

a  a  a 

DOCS1_3  0.133"  0.135" 

(0.023)  (0.024) 

DOCS3_5  0.168*  0.169" 

(0.031)  (0.031) 
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Table  2-4  (Cont'd):  Regression  Equations  Examining  Whether  a  Proxy  Respondent 
Provided  PPCIS  Cost  Data 


Independent 
Variable 


Model  1 


Model  2 


Model  3 


DOCS5  8 


0.218" 
(0.041) 


0.222" 
(0.042) 


DOCS8  20 


0.361" 
(0.040) 


0.366 
(0.040) 


DOCS21 


0.370" 
(0.053) 


0.388" 
(0.054) 


M  SPEC 


CORP 


0.004 
(0.033) 

0.094" 
(0.019) 


0.003 
(0.033) 

0.096" 
(0.019) 


HOW  LATE 


a 
a 


HOW  LONG 


a 
a 


CATI 


Constant 


Adjusted  R-Square 


-0.112 
(0.028) 

0.595" 
(0.111) 

21.16 
0.13 


0.494" 
(0.083) 

21.83 
0.12 


0.531" 
(0.086) 

12.72 
0.04 


Source:  Project  HOPE  analysis  of  1988  PPCIS  data. 

Notes:  Mean  values  of  dependent  and  independent  variables  are  displayed  in  Table  2-3.  Statistical  significance 
of  the  regression  coefficient  is  indicated  as  follows:  **  at  alpha  level  0.01;  *  at  alpha  level  0.05. 
•  Value  less  than  0.005. 
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models.  On  average,  physicians  in  the  largest  SMS  As  were  16  percent  (six  percentage 
points)  less  likely  to  use  a  proxy  than  physicians  in  smaller  SMSAs  ~  0.39  probability  in  the 
largest  SMSAs  versus  0.45  in  small  SMSAs  (Table  2-5).  Physicians  in  non-metropolitan 
areas  were  the  most  likely  to  use  a  proxy  (likelihood  of  0.47,  controlling  for  other  factors.) 
It  is  interesting  to  note  that  results  from  the  Marder-Thran  analysis  of  the  AMA's  SMS 
survey  are  similar.  Physicians  in  the  largest  SMSAs  are  less  likely  to  delegate  expense 
questions  to  proxy  respondents  than  physicians  in  non metropolitan  areas. 

The  impact  of  work  hours  ~  the  other  measure  of  "physician  busyness"  -  is  not 
consistent  with  prior  expectations  and  the  direct  univariate  relationship  displayed  in  Table  2- 
1 .  After  controlling  for  physician  and  practice  characteristics,  hours  has  a  negative  affect  on 
proxy  use  (Table  2-4).  On  the  other  hand,  the  effect  is  very  small  in  magnitude.  A  large 
increase  in  the  length  of  the  work  week  from  the  sample  average  of  58.6  hours  (Table  2-3)  to 
70  hours  would  only  reduce  the  probability  of  proxy  use  by  0.006  percentage  points  (from 
0.429  to  0.423). 

Practice  size  is  directly  related  to  proxy  use,  as  hypothesized.  Predicted  estimates  of 
the  probabilities  of  proxy  use  increase  monotonically  with  practical  size.  Practices  with 
between  one  and  three  full-time-equivalent  physicians  are  37  percent  more  likely  to  use  a 
proxy  than  solo  practice  physicians  -  a  predicted  likelihood  of  0.48  versus  0.35  for  solo 
practices  (Table  2-5).  Physicians  of  the  largest  practices  (practices  with  over  eight  full-time 
physicians)  are  over  twice  as  likely  to  use  proxies  as  solo  practitioners.  Again,  comparisons 
with  the  Marder-Thran  analysis  are  of  interest.  In  their  model,  practice  size  is  measured 
using  a  categorical  variable  that  distinguished  solo  from  target  practices.  Practice  size 
doesn't  affect  use  of  proxy  direcdy,  but  does  affect  proxy  use  via  its  effect  on  the  predicted 
physicians' s  wage.  Solo  practice  physicians  have  smaller  wages  than  physicians  in  larger 
practices.  As  wages  are  directly  related  to  proxy  use,  Marder  and  Thran  also  find  that  solo 
practitioners  are  less  likely  to  use  proxy  respondents  than  physicians  in  larger  practices. 
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Table  2-5:    Predicted  Likelihoods  of  Proxy  Use  by  Geographic  Location  and 
Practice  Size 


Estimated 

Likelihood*  Ratiob 


Geographic  Location: 

Metropolitan  area, 

more  than  1  million  households  0.39  0.87" 
Metropolitan  area, 

less  than  1  million  households  (R)  0.45  1.00 

Non-metropolitan  area  0.47  1 .04 


Practice  Size: 

Solo  (R)  0.35  1.00 

1-3  physicians  0.48  1.37' 

3-5  physicians  0.51  1.46* 

5-8  physicians  0.56  1.60" 

8-20  physicians  0.71  2.03" 

More  than  20  physicians  0.72  2.06* 


Source:  Analysis  of  1988  PPCIS  data  by  Project  HOPE. 

Notes:  '  Estimates  based  on  Model  3  estimates.  Table  2-4;  variables  other  than  geographic  location  and  practice 
size  were  assigned  their  mean  values. 

b  Calculated  as  the  estimated  likelihood  divided  by  the  likelihood  for  the  reference  category  (R).  An  asterisk 
indicates  that  the  regression  efficient  for  the  category  (Table  2-4,  Model  l)is  statistically  significant  at  the  0.05 
level  of  confidence. 
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Variables  indicating  whether  the  practice  is  incorporated  and  whether  it  is  a 
multispecialty  practice  were  included  in  the  model  as  measures  of  practice  complexity. 
Incorporated  physicians  are  nine  percentage  points  more  likely  to  use  a  respondent  than 
unincorporated  physicians  (Table  2-4).  While  univariate  statistics  suggested  that  physicians 
in  multispecialty  groups  were  more  likely  to  rely  on  proxy  respondents  (Table  2-2),  there  is 
no  evidence  of  this  association  in  the  linear  probability  models. 

Two  measures  of  the  timing  of  data  collection  activities  are  included  in  the  full 
regression  model  (Table  2-4,  Model  1)  along  with  the  CATI  dummy  variable.  As  noted 
above,  the  variable  HOW_LATE  is  a  measure  of  when  data  collection  activities  with  the 
practice  began.  The  variable  HOW_LONG  was  constructed  as  a  measure  of  elapsed  time 
before  data  collection  at  the  practice  was  completed.  Only  the  CATI  indicator  variable  was 
statistically  significant  at  the  conventional  level  of  0.05.  Use  of  a  proxy  respondent  and 
CATI  procedures  were  negatively  related,  ceterus  paribus.  One  explanation  is  that  when 
proxy  respondents  are  used,  the  CATI  mode  is  discontinued  for  pencil-and-paper  recording 
of  practice  cost  responses.  It  should  be  noted  that  the  effect  of  the  CATI  mode  was  one  of 
the  significant  design  effects  noted  in  an  earlier  report  on  the  PPCIS  (Project  HOPE,  1991). 

2.5  Implications 

Nonphysician  proxy  respondents  are  often  used  to  provide  information  on  practice 
costs  as  part  of  physician  surveys  that  are  used  to  affect  policy.  The  PPCIS  has  been  used  to 
estimate  the  Medical  Economic  Index  (MEI),  an  index  that  has  been  used  to  adjust  Medicare 
fees  over  time.  Approximately  one-half  of  PPCIS  practice  cost  respondents  were  proxies. 
In  a  similar  fashion,  many  respondents  to  the  AMA's  SMS  surveys  -  used  to  develop  the 
practice  expense  component  of  the  new  Medicare  Resource-Based  Relative  Value  Scale  --  are 
proxy  respondents. 

This  work  was  motivated  by  the  finding  of  a  design  effect  associated  with  the  use  of 
proxy  respondents  in  the  PPCIS.  In  particular,  proxy  respondents  reported  total  practice 

2-19 


costs  that  were  60  percent  of  costs  reported  by  physicians  (Project  HOPE,  1991).  Results  of 
this  analysis  confirm  that  systematic  differences  in  practices  exist  between  practices  with 
physician  and  nonphysician  respondents.  For  the  PPCIS,  proxy  respondents  were  more 
likely  to  provide  information  on  practice  costs  in  larger  practices,  in  practices  in  non- 
metropolitan  and  smaller  metropolitan  areas,  and  in  practices  that  are  incorporated.  Marder 
and  Thran  (1989)  observed  consistent  relationships  between  geographic  area,  practice  size, 
and  use  of  proxy  respondents  in  the  AMA's  SMS:  proxy  respondents  were  used  less 
frequently  in  large  metropolitan  areas,  and  the  value  of  time  of  solo  (as  opposed  to  group) 
physicians  is  less,  which  decreases  the  likelihood  that  they  will  designate  a  proxy  respondent. 

A  key  question  is  whether  data  from  both  types  of  respondents  are  valid.  If  so,  the 
design  effect  may  be  an  artifact.  Type  of  respondent  may  be  associated  with  one  or  more 
practice  characteristics  that  are  not  otherwise  controlled  for  in  the  estimated  cost  function  — 
an  empirical  specification  that  frequently  is  estimated  by  economists.  For  example,  the 
design  effect  may  be  an  indication  that  larger  practices  are  more  efficient  than  smaller 
practices.  In  particular,  larger  practices  are  more  likely  to  hire  personnel  to  assist  with 
financial  matters  (and  these  persons  served  as  proxy  respondents).  At  best,  however,  this 
explanation  is  only  a  partial  explanation  because  the  practice  differences  identified  above  do 
not,  by  themselves,  account  fully  for  the  design  effect.  First,  only  a  relatively  small  amount 
of  the  variation  in  respondent  type  is  explained  by  the  regressions  in  this  chapter.  The  model 
with  the  largest  number  of  variables  only  explains  1 3  percent  of  the  variation  in  respondent 
type.  Second,  the  most  important  determinants  of  respondent  type  —  practice  size  and 
metropolitan  area  ~  had  statistically  significant,  independent  effects  on  cost  in  the  cost 
equations  that  exhibited  the  design  effect.  Thus,  the  design  affect  most  likely  reflects 
additional,  independent  factors  as  well  as  those  observed  above. 

If,  on  the  other  hand,  practice  cost  data  from  physicians  and  proxy  respondents  are 
not  equally  valid,  the  observed  design  effect  may  be  real.  The  systematic  differences  leading 
to  choice  of  respondent  may  introduce  bias  in  practice  costs  estimates  from  the  survey.  In 
this  case,  special  survey  design  procedures  may  be  necessary  to  ensure  that  physicians  and 
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proxy  respondents  are  providing  valid  data.  More  information  on  the  nature  and  quality  of 
data  provided  by  physicians  and  proxy  respondents  is  needed. 
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CHAPTER  3: 


SOME  EVIDENCE  ON  THE  RELATIONSHIP  BETWEEN 
PROXY  USE  AND  DATA  QUALITY 

3.1  Overview 

There  are  at  least  two  explanations  that  would  account  for  the  design  effect  associated 
with  type  of  PPCIS  respondent.  As  noted  in  Chapter  2,  one  explanation  is  that  there  are 
systematic  differences  between  practices  for  which  physicians  versus  proxy  respondents 
provide  practice  cost  data,  and  that  these  differences,  rather  than  type  of  respondent  per  se, 
account  for  differences  in  costs.  A  second  explanation  is  that  there  are  differences  in  the 
validity  of  data  provided  by  physicians  and  proxy  respondents.  Unfortunately,  there  are  no 
data  that  permit  comparisons  of  reported  cost  information  provided  by  physicians  and  proxy 
respondents  of  the  same  practices. 

In  this  chapter,  data  from  the  1988  PPCIS  are  used  to  begin  to  address  differences  in 
the  nature  of  data  reported  by  physicians  and  proxy  respondents.  The  central  question  is 
whether  physician  or  proxy  respondents  are  more  knowledgeable  sources  of  practice  cost 
information.  Answers  to  this  question  potentially  affect  the  design  of  future  surveys.  Two 
indicators  of  knowledge  are  considered:  the  incidence  of  "Don't  Know"  responses  to  cost 
questions,  and  the  ability  or  willingness  of  the  respondent  to  provide  detailed  cost 
information  as  requested  by  the  PPCIS.  How  these  indicators  are  used  is  described  in  the 
next  section.  Findings  and  implications  are  discussed  in  sections  3.3  and  3.4. 
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3.2     Framework  and  Methods 


A  physician  might  designate  a  proxy  respondent  if  he  or  she  is  simply  too  busy  to 
respond  to  detailed  questions  on  costs.  Ideally,  the  designated  proxy  respondent  is  the  best 
source  of  information  on  costs,  but  in  fact,  the  respondent  may  or  may  not  be  more 
knowledgeable  on  cost  issues  than  the  physician.  Small  practices  are  less  likely  to  have 
practice  managers  with  focused  cost-accounting  responsibilities  (Scheffler,  1975).  By 
contrast,  larger  groups  are  able  to  hire  specialized  labor  to  assume  managerial  and  financial 
responsibilities.  A  second  reason  a  physician  might  delegate  cost  questions  is  that  the 
physician  feels  that  the  proxy  respondent  would  be  more  knowledgeable  and  readily  able  to 
provide  the  cost  information.  For  example,  a  physician  in  a  multi-specialty  group  with  eight 
physicians  may  not  have  detailed  information  on  costs  of  the  practice,  especially  costs  that 
are  incurred  by  or  on  behalf  of  other  physicians  in  the  group.  Indeed,  empirical  analysis  in 
the  preceding  chapter  indicates  that  a  physician's  busyness  and  practice  size  are  directly 
related  to  the  likelihood  that  a  proxy  respondent  will  be  used. 

Data  from  the  1988  PPCIS  can  be  used  to  examine  whether  proxy  respondents  are 
more  knowledgeable  concerning  costs  than  physician  respondents.  It  is  hypothesized  that 
proxy  respondents  are  more  knowledgeable  in  larger  practices.  The  cost  structures  of  larger 
practices  are  more  complex.  In  these  settings,  cost  information  is  relatively  difficult  and 
costly  for  physicians  to  obtain.  In  smaller  practices,  however,  observed  differences  in 
knowledge  may  not  be  as  apparent;  proxy  respondents  are  more  likely  to  engage  in  a  variety 
of  administrative  activities  and  less  likely  to  function  only  as  business  managers. 

The  PPCIS  contains  questions  on  the  major  components  of  costs  facing  the  practice. 
These  include  wages  paid  to  physicians  and  non-physician  employees,  the  costs  of  occupancy 
(primarily  rent  and  depreciation  of  office  space),  and  the  costs  of  medical  equipment  and 
supplies.  The  survey  also  includes  questions  on  amounts  of  deferred  compensation  and 
fringe  benefits  for  practice  physicians  and  non-physician  employees.  The  PPCIS  analysis  file 
indicates  whether  personnel  costs  -  including  wages,  deferred  compensation,  and  fringe 


benefits  -  were  reported  separately  by  the  respondent  or  whether  the  respondent  reported 
only  the  less-detailed  cost  aggregates. 

Two  indices  of  respondent  knowledge  were  calculated  from  the  survey  data.  "Don't 
Know"  (DK)  response  rates  for  physicians  and  proxy  respondents  were  calculated  for  PPCIS 
questions  on  the  most  important  determinants  of  practice  costs  —  wages,  costs  of  medical 
equipment,  and  supply  costs.  Rates  were  calculated  by  dividing  the  number  of  DK  responses 
by  the  number  of  physicians  participating  in  the  survey,  less  the  number  that  refused  to 
answer  the  question.1  "Don't  Know"  rates  were  also  calculated  for  deferred  compensation 
and  fringe  benefit  amounts.  In  the  calculation  of  these  rates,  a  DK  response  indicates  that 
the  respondent  was  unable  to  report  the  cost  amount,  either  separately  or  as  part  of  a  total 
cost  of  compensation.  "Don't  Know"  response  rates  were  calculated  for  physician  and  proxy 
respondents,  by  practice  size.  Two  size  categories  were  used:  small  practices,  defined  as 
those  with  three  or  fewer  full-time-equivalent  physician  "owners,"  and  larger  practices  with 
more  than  three  physician  owners.2  Differences  in  DK  rates  by  type  of  responder  and 
practice  size  have  survey  design  and  health  policy  implications  as  discussed  below. 

A  second  index  of  respondent  knowledge  is  whether  the  respondent  was  able  to 
separate  personnel  costs  into  their  wage,  deferred  compensation,  and  fringe  benefit 
components.  The  rate  at  which  respondents  were  able  to  provide  such  detail  (hereafter,  the 
"detail  rate")  was  calculated  by  dividing  the  number  of  respondents  who  provided  the 
detailed  information  by  the  total  number  of  physicians  participating  in  the  survey,  less  those 
who  refused  to  answer  the  series  of  compensation  questions.  The  detail  rate  was  also 
calculated  for  both  types  of  respondents,  by  practice  size. 


'Rates  of  refusal  do  not  vary  by  type  of  respondent.  Item  refusal  rates  for  several  cost  questions  are  displayed 
in  the  Appendix. 

:  As  in  Chapter  2,  practice  size  is  defined  as  the  number  of  full-time-equivalent  physician  owners  or  partners 
of  the  practice. 
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3.3  Findings 


3.3.1  Don't  Know  Rates 

Don't  Know  rates  from  the  PPCIS  can  be  used  to  examine  (i)  relative  differences 
between  proxy  and  physician  response,  given  practice  size;  (ii)  differences  in  proxy  response 
by.  practice  size;  and  (iii)  differences  in  physician  response  by  practice  size. 

Relative  Differences  in  Proxy  and  Physician  Response.  Proxy  respondents  in  large 
practices  are  more  knowledgeable  (or  at  least  more  willing  to  provide  the  requested 
information)  than  physician  respondents  (Table  3-1).  Reported  DK  rates  for  all  eight  data 
items  are  greater  for  physicians  than  for  proxy  respondents  of  practices  with  more  than  three 
physicians,  and  are  statistically  significant  at  an  alpha  level  of  0.05  or  better.  The  DK 
response  rate  for  physician  wages  (3.7  percent)  was  seven  times  greater  than  the  rate  for 
proxy  respondents  (0.5  percent).  All  proxy  respondents  in  the  analysis  sample  were  able  to 
report  non-physician  employee  costs,  while  3.0  percent  of  physicians  were  not.  Four  times 
as  many  physicians  as  proxy  respondents  were  not  able  (or  willing)  to  report  medical  supply 
expenses  (6.4  percent  versus  1.6  percent). 

At  the  same  time,  there  is  no  evidence  indicating  that  proxy  respondents  are  more 
knowledgeable  than  physicians  in  small  practices.  DK  rates  for  proxy  respondents  are  at 
least  as  large  as  physician  rates  for  four  of  the  eight  data  items  displayed  in  Table  3- 1 . 
Proxy  respondents  in  small  practices  appear  less  willing  or  able  to  provide  information  on 
remuneration  of  physicians.  More  than  twice  as  many  DKs  were  reported  in  data  collection 
from  proxy  respondents  (2.4  percent)  as  from  physicians  themselves  (1.3  percent)  in  response 
to  the  question  on  the  amount  of  deferred  compensation  for  physicians. 

Differences  in  Proxy  Response  by  Practice  Size.  Don't  Know  rates  for  proxy 
respondents  differ  by  practice  size.  Proxy  respondents  in  large  practices  are  more  likely  to 
provide  information  than  respondents  from  smaller  practices.  Don't  Know  rates  for  proxy 
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Table  3-1:    "Don't  Know"  (DK)  Rates  for  Physician  and  Respondents, 
by  Size  of  Practice 


Physicians  <_3  Physicians  >  3 

Physician  Proxy  Physician  Proxy 


Physician  costs 


wages  1.3  2.4  3.7C  0.5ab 

deferred  compensation  2.2  4.5a  6.3°  1 .9ab 

fringe  benefits  3.9  4.1  6.8C  1.7ab 


Office  employee  costs 

wages  1.4  1.8  3.0C  0.0a 

deferred  compensation  2.9  2.9  6.0C  1 .6" 


fringe  benefits  4.0  3.6  3.9  1.4 
Medical  equipment  costs 

depreciation  10.5  9.2  9.4  7.4 

Medical  supply  expenses  5.0  4.3  6.4  1.6 


a.b 


Source:  Analysis  of  1988  PPCIS  data  by  Project  HOPE. 

Note:     "  Different  from  rate  for  physician  respondents  in  practices  of  same  size  category,  alpha  =  0.05. 
b    Different  from  rate  for  proxy  respondents  in  small  practices,  alpha  =  0.05. 
c    Different  from  rate  for  physician  respondents  in  small  practices,  alpha  =  0.05. 
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respondents  of  large  practices  are  consistently  less  than  in  small  practices.  Differences  are 
statistically  discernable  for  the  questions  of  physician  costs,  non-physician  fringe  benefits, 
and  the  costs  of  medical  supplies.  The  likelihood  that  a  proxy  respondent  in  a  small  practice 
was  unwilling  or  unable  to  provide  data  on  physician  wages  was  more  than  four  times  greater 
than  for  the  average,  large  practice  proxy  respondent. 

Differences  in  Physician  Response  by  Practice  Size.  The  physician  appears  to  be  a 
more  knowledgeable  source  of  cost  data  in  smaller  practices  than  in  larger  practices  ~ 
especially  for  data  on  employee  costs.  The  DK  rate  of  physician  respondents  in  small 
practices  was  1.3  percent  for  physician  wages,  versus  3.7  percent  in  large  practices;  the  DK 
rate  for  wages  of  office  employees  was  twice  as  large  for  physicians  in  large  practices  (3.0 
percent)  as  for  physicians  in  small  practices  (1.4  percent). 

3.3.2  Detail  Rates 

The  second  measure  of  respondent  knowledge  is  the  "detail  rate"  --  a  measure  of  the 
extent  to  which  a  respondent  was  able  (and  willing)  to  report  cost  information  in  a 
disaggregated  form  as  requested  by  the  interviewer.  The  availability  of  data  describing  the 
components  of  physician  and  non-physician  employee  costs  is  presumed  to  be  an  indicator  of 
respondent  knowledge;  more  detailed  information  on  the  composition  of  employee  costs  is 
also  presumed  to  be  of  more  potential  policy  usefulness  than  the  aggregate  of  total  labor 
costs. 

Evidence  on  differences  in  detail  rates  between  respondents  and  by  practice  size  is 
mixed.  For  physician  compensation,  no  differences  between  the  detail  rates  of  physicians 
and  proxy  respondents  in  large  or  small  practices  are  observed  (Table  3-2).  Furthermore, 
rates  for  proxy  respondents  are  similar  in  large  and  small  practices,  and  detail  rates  of 
physicians  are  comparable  in  large  and  small  practices. 
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Table  3-2:    Detail  Rates  for  Physician  and  Proxy  Respondents, 
by  Size  of  Practice 


Physicians  <_3  Physicians  >  3 

Physician  Proxy  Physician  Proxy 


Data  Item 


Physician  fringe  benefits 
and  deferred  compensation     79.9  78.6  73.9  76.6 


Employee  fringe  benefits 
and  deferred  compensation     80.2  82.4  76.7  95.28b 


Source:  Analysis  of  1988  PPCIS  data  by  Project  HOPE. 

Note:     '  Different  from  rates  for  physician  respondents  in  practices  of  same  size  category,  alpha  =  0.05. 
b  Different  from  rate  for  proxy  respondents  in  small  practices,  alpha  =  0.05. 
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For  nonphysician  employee  costs,  however,  some  differences  in  detail  rates  are 
apparent  (Table  3-2).  Proxy  respondents  in  large  practices  were  most  able  to  provide 
detailed  information  on  employee  costs.  Ninety-five  percent  of  proxy  respondents  in  large 
practices  provided  full,  disaggregated  employee  cost  information,  while  only  77  percent  of 
physicians  in  large  practices  responded  with  full  information.  Relatively  more  proxy 
respondents  provided  complete  information  on  employee  costs  in  larger  practices  than  in 
smaller  practices  (95  percent  versus  82  percent).  Point  estimates  of  detail  rates  for 
physicians  in  small  practices  exceed  those  in  large  practices,  but  are  not  different  at  the  0.05 
level  of  statistical  significance.  The  employee  cost  detail  rate  for  proxy  respondents  in 
smaller  practices  (82  percent)  does  not  significantly  differ  from  the  rate  for  physicians  (80 
percent). 

3.4  Implications 

The  central  question  addressed  above  is  whether  physician  or  proxy  respondents  are 
more  "knowledgeable"  about  practice  costs.  Two  measures  of  knowledge  were  considered: 
the  fraction  of  respondents  who  provided  responses  to  questions  about  key  components  of 
practice  costs  —  in  other  words,  the  fraction  of  respondents  who  were  able  and  willing 
respondents  —  and  the  fraction  of  respondents  who  were  able  and  willing  to  provide  details 
on  the  breakdowns  of  physician  and  nonphysician  compensation.  Several  conclusions  follow, 
based  on  statistical  analysis  of  the  two  measures. 

First,  proxy  respondents  are  more  knowledgeable  than  physician  respondents  in  large 
practices  (practices  with  more  than  three  physicians).  One  explanation  for  this  finding  is  that 
the  typical  large  practice,  proxy  respondent  is  a  person  with  financial  responsibilities.  Large 
practices  are  large  enough  to  support  the  hiring  of  specialized  financial  personnel  whose 
responsibilities  require  a  general  knowledge  of  the  entire  practice.  This  explanation  is 
supported  by  the  second  conclusion:  proxy  respondents  in  large  practices  are  more 
knowledgeable  than  proxy  respondents  in  small  practices.  Proxy  respondents  of  small 
practices  are  likely  to  serve  a  number  of  functions,  including  general  office  personnel 
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management  and  clerical  tasks.    The  final  conclusion  is  not  independent:  in  general, 
physician  respondents  in  small  practices  appear  to  be  more  knowledgeable  about  practice  cost 
matters  than  their  colleagues  in  large  practices. 

These  conclusions  have  implications  on  the  design  of  practice  cost  surveys.  Research 
is  needed  to  directly  determine  whether  physician  and  proxy  respondent-provided  data  are 
valid,  and  to  what  extent  proxy  respondents  are  more  likely  to  have  financial  responsibilities 
within  the  practice.  Evidence  from  this  chapter  suggests  that  proxy-provided  information 
from  large  practices  may  be  better  than  information  from  the  physician  respondent.  If  so, 
then  practice  cost  surveys  of  physicians  in  large  practices  should  be  designed  specifically  to 
collect  data  from  proxy  respondents.3  Physicians  would  support  this  change,  to  the  extent 
that  busy  physicians  are  the  most  reluctant  to  participate  in  cost  surveys.  Failure  to  do  so 
may  introduce  biases  that  affect  public  policies,  such  as  distorted  practice  expense  payment 
levels.  Such  a  design  change  would  be  especially  important  as  the  average  size  of  medical 
practices  continues  to  increase  and  proposed  health  reforms  tend  to  encourage  the  growth  of 
physician  groups. 


3This  assumes  that  a  survey  is  the  mechanism  of  choice  for  collecting  needed  practice  cost  information.  This 
assumption  is  challenged  in  the  next  chapter. 
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CHAPTER  4: 


SHOULD  SURVEY  DATA  BE  USED  TO  ESTIMATE 
PHYSICIAN  PRACTICE  COSTS? 

4.1  Background 

Accurate  estimation  of  the  components  of  physician  practice  costs  has  traditionally 
been  an  area  of  concern  to  health  policy  makers.  Practice  expense  ~  defined  in  physician 
payment  reform  legislation  as  the  total  of  practice  costs,  less  the  costs  of  physician  labor  and 
malpractice  premiums  —  is  a  critical  component  in  the  calculation  of  payments  that 
physicians  receive  for  providing  care  to  the  Medicare  population.  An  estimated  40  percent 
of  payments  under  the  new  Medicare  Fee  Schedule  is  based  on  practice  cost  survey  data 
(Federal  Register.  1991).  The  Health  Care  Financing  Administration  has  conducted  the 
PPCIS  periodically,  and  its  data  have  been  used  to  update  payments  to  physicians.  If 
practice  costs  are  not  accurately  reported,  payment  levels  may  be  inaccurate. 

Over  the  last  ten  years,  there  has  been  a  growing  body  of  primarily  unpublished 
research  which  evaluates  the  problems  associated  with  conducting  physician  surveys.    In  this 
chapter,  some  of  these  findings  are  reviewed  and  synthesized.  Should  surveys  of  physicians 
be  used  to  measure  practice  costs?  Should  other  ways  to  obtain  information  on  practice  costs 
be  considered? 

In  the  next  section,  the  ways  in  which  sources  of  error  relate  to  physician  practice 
cost  surveys  are  reviewed.  Rates  of  unit  and  item  nonresponse  over  time  are  reviewed. 
While  there  is  a  trend  toward  lower  unit  response  rates  and  higher  item  response  rates, 
cumulative  response  rates  have  fallen  in  recent  years.  Various  potential  sources  of 
measurement  error  are  also  reviewed.  The  chapter  concludes  by  questioning  whether 
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physician  surveys  should  continue  to  be  used  as  a  source  of  practice  cost  data;  other  methods 
that  could  result  in  more  accurate  measures  of  practice  costs  are  suggested. 

4.2  Types  of  Error 

The  framework  for  this  discussion  is  the  topology  of  bias  used  by  Groves  (1989). 
Biases  in  practice  cost  surveys  may  arise  from  four  major  types  of  errors:  sampling  error, 
coverage  error,  nonresponse  error,  and  measurement  error. 

4.2.1  Sampling  Error 

Sampling  error  is  a  function  of  sample  size  and  design  effects.  It  refers  to  the  level 
of  variance  of  estimates  produced  from  survey  data.  Among  the  key  estimates  from  practice 
cost  surveys  are  cost  estimates  by  specialty.  Estimates  are  often  based  on  cell  sizes  of 
between  100  and  200  per  specialty.  When  the  analyses  require  further  breakdowns,  such  as 
estimates  for  urban  and  rural  comparisons,  estimates  have  relatively  high  variances  due  to 
sampling  error.  Although  sampling  error  does  not  in  itself  lead  to  bias,  practice  cost 
estimates  may  suffer  from  a  lack  of  precision  because  of  sampling  error. 

4.2.2  Coverage  Error 

Coverage  error  occurs  when  members  of  the  target  population  have  no  chance  of 
selection  into  the  sample.  Of  the  four  major  types  of  bias  in  the  Groves'  topology,  coverage 
error  is  probably  the  least  important  source  of  bias  for  physician  practice  cost  surveys. 
Sample  selection  for  the  major  surveys  that  have  focused  on  practice  costs,  including  the 
PPCIS  and  surveys  sponsored  by  the  Physician  Payment  Review  Commission  (PPRC),  and 
the  American  Medical  Association  (AM A),  is  based  on  the  AMA's  Physician  Master  File  as 
the  sampling  frame.  This  file  is  designed  to  include  the  universe  of  all  practicing  physicians. 
Although  there  has  been  relatively  little  formal  evaluation  of  the  AMA  Masterfile,  it  is 
continuously  updated.  Researchers  generally  have  assumed  coverage  bias  to  be  small.  The 
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AMA  Masterfile  is  regarded  as  the  "model"  for  development  of  other  list  frames  for  different 
types  of  medical  providers  (Schoenman,  1988). 

4.2.3  Nonresponse  Error 

Nonresponse  error  is  a  potentially  serious  source  of  error  arising  from  surveys  of 
practice  costs.  There  are  two  types  of  nonresponse  error.  Unit  nonresponse  error  occurs 
when  physicians  refuse  to  participate  in  any  part  of  the  survey;  item  nonresponse  error  may 
occur  when  the  physician  fails  to  complete  critical  portions  of  the  survey,  as  when  physicians 
refuse  to  provide  data  on  certain  components  of  practice  costs.  The  total  level  of 
nonresponse  error  is  a  multiplicative  function  of  unit  nonresponse  and  item  nonresponse 
rates. 

Unit  Response  Rates.  Table  4-1  displays  data  on  the  response  rates  obtained  by 
economic  surveys  of  physicians  sponsored  by  the  federal  government.  Two  federally 
sponsored  surveys  have  obtained  response  rates  in  excess  of  70  percent:  the  Physician's 
Practice  Survey  and  the  Patients'  Identified  Physician  Survey  of  the  National  Medical 
Expenditure  Survey  (NMES)  pretest.  Both  of  these  surveys  were  designed  as  significantly 
shortened  versions  of  the  PPCIS;  response  rates  of  69  and  61  percent  were  obtained  in  1969 
and  1988,  respectively. 

Fifteen  years  ago,  the  American  Medical  Association  was  obtaining  response  rates  on 
physician  practice  surveys  that  were  significantly  below  those  obtained  by  the  federal 
government.  This  pattern  has  reversed.  During  the  period  between  1967  and  1977,  response 
rates  for  the  AMA's  Periodic  Survey  of  Physicians  declined  from  70.4  percent  to  48.7 
percent.  After  viewing  this  decline,  the  AMA  redesigned  the  survey  and  began  fielding  the 
Socioeconomic  Monitoring  System  (SMS)  surveys  which  relied  primarily  on  the  use  of  the 
telephone.  Response  rates  have  steadily  increased  from  60.3  percent  in  1988  to  72.1  percent 
in  1989,  in  contrast  to  government  response  rates  of  61  and  62  percent  in  the  late- 1980s. 
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Table  4-1 :    Response  Rates  of  Government-Sponsored  Surveys  of 
Practice  Costs 


Year  Title  Sponsor 

1977  Physician's  Practice  Survey  NCHSR 

1983  Physicians' Practice  Cost  HCFA 

and  Income  Survey  (PPCIS) 

1986  Patients'  Identified  Physician  AHCPR 

Survey  (NMES  Pretest) 

1988  Physicians' Practice  Cost  and  HCFA 

Income  Survey  (PPCIS) 

1988  National  Survey  of  Physicians  PPRC 

(NSOP) 


Response  Rate 
74 
69 

89 

61 

62 
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In  recent  years,  response  rates  to  the  AMA  surveys  have  declined  from  72.1  in  1989 
to  64.4  percent  in  1992.  During  the  last  five  years,  the  federal  government  has  not 
conducted  any  national  practice  cost  surveys.  The  decline  in  response  rates  to  the  AMA 
surveys  during  the  1980's,  however,  should  be  of  concern  to  the  government. 

The  reasons  for  this  decline  are  not  clear.  Hostility  to  federal  policies  regarding 
physicians  is  sometimes  seen  as  a  obstacle  to  obtaining  high  response  rates  on  federal 
surveys,  but  Berk  et  al.  (1989)  found  no  evidence  that  this  was  a  major  factor.  Another 
possible  reason  relates  to  changes  in  physicians'  practices  themselves.  Physicians'  practices 
have  grown  larger,  and  individual  physicians  may  be  under  increased  pressure  to  reduce  time 
spent  on  administrative  activities,  including  participation  in  surveys.  In  some  cases, 
physicians  may  lack  the  autonomy  to  release  practice  information  without  approval  from  the 
group's  or  corporation's  governing  body.  This  helps  explain  the  observed  relationships 
between  use  of  proxy  respondents,  practice  size,  and  physician  busyness  reported  in  Chapter 
2. 

While  physician  surveys  generally  achieve  lower  response  rates  than  other  types  of 
social  surveys,  it  is  not  always  clear  what  effect  nonresponse  error  has  on  final  survey 
estimates.  Several  evaluations  of  physician  surveys  indicate  that  the  addition  of  late 
responders  did  not  lead  to  a  more  representative  sample  (Berk,  1985;  Guadagnoli  & 
Cunningham,  1989;  Sobal  &  Ferentz,  1989).  While  researchers  should  certainly  try  to 
improve  response  rates,  they  should  be  cognizant  of  the  effect  such  efforts  will  have  on 
overall  bias.  As  Berry  &  Kanouse  (1987)  note,  "A  higher  response  rate  will  not  always 
result  in  a  more  representative  sample.  In  fact,  some  methods  of  boosting  response  rates 
may  do  so  at  the  expense  of  introducing  further  bias  because  some  methods  of  boosting 
response  rates  will  appeal  more  to  some  type  of  respondents  than  others"  (p.  112). 

Item  Response  Rates.  Despite  the  trend  toward  lower  unit  response  rates,  there  is 
evidence  to  suggest  that  item  response  rates  are  improving.  This  is  significant  -  too  often, 
researchers  focus  only  on  levels  of  unit  nonresponse  even  though  "missing  data  on  specific 
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items  is  much  more  a  part  of  the  measurement  process  than  is  unit  nonresponse"  (Groves, 
1989,  p.  156). 

Item  response  rates  on  the  1988  National  Survey  of  Physicians  (NSOP)  were  much 
higher  than  on  the  1983  PPCIS.  The  NSOP  had  a  total  of  approximately  250  questions. 
About  64  percent  of  the  questions  had  an  item  nonresponse  rate  of  less  than  10  percent, 
whereas  only  10  percent  of  the  questions  had  item  nonresponse  rates  that  exceeded  30 
percent  (Berk  et  al.,  1988). 

In  the  NSOP,  item  nonresponse  rates  were  generally  low  for  questions  on  Medicare 
assignment,  practice  arrangements,  and  most  components  of  practice  costs.  High  item 
nonresponse  rates  were  obtained  for  questions  relating  to  equipment  purchase  and 
maintenance  costs.  For  example,  more  than  40  percent  of  respondents  did  not  attempt  to 
report  the  purchase  costs  of  an  X-ray  machine,  lease  payments  for  maintenance  costs  of  X- 
ray  machines,  or  maintenance  contracts  for  EKG  machines  and  mammography  equipment. 

Comparison  of  the  1988  and  1983  PPCIS  surveys  also  indicates  significant 
improvement  in  item  response  rates  (Project  HOPE,  March,  1991).  In  the  1988  PPCIS,  65 
percent  of  all  questions  had  item  nonresponse  rates  of  between  zero  and  five  percent.  This  is 
a  dramatic  improvement  over  the  1983  PPCIS,  in  which  only  20  percent  of  the  questions 
achieved  this  low  level  of  item  nonresponse.    There  was  considerable  improvement  in  rates 
for  questions  concerning  malpractice  insurance  for  all  practice  members,  tax  deductibility  for 
professional  costs,  and  questions  related  to  the  depreciation  of  equipment. 

These  findings  suggest  that  there  is  an  inverse  relationship  between  unit  nonresponse 
and  item  nonresponse.  It  is  possible  that  as  pressures  to  obtain  participation  from  reluctant 
providers  increase,  data  quality  deteriorates.  In  examining  data  from  the  1977  Physician's 
Practice  Survey,  Berk  (1985)  found  that  as  the  survey  period  went  on,  there  were  significant 
decreases  in  the  percentage  of  cases  in  which  the  interviewer  rated  the  cooperation  of  the 
respondent  as  good  or  excellent.  There  also  was  a  decline  in  the  number  of  respondents  who 
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answered  the  physicians'  total  income  question  ~  one  of  the  key  data  items  in  the  survey  — 
during  the  field  period. 

Cumulative  Response  Rates.  It  appears  that  over  the  last  decade,  unit  response  rates 
have  declined  while  item  response  rates  have  improved.  More  physicians  are  declining  to 
participate  in  surveys,  but  those  that  do  participate  are  providing  more  complete  information. 
Cumulative  response  rates  can  be  obtained  by  multiplying  the  survey  participation  rate  by  the 
percentage  of  valid  responses  received  on  each  question.  Table  4-2  presents  cumulative 
response  rates  for  the  AMA's  SMS  core  surveys  conducted  between  1982  and  1992. 

During  the  six-year  period  between  1982  and  1988,  cumulative  response  rates  clearly 
improved  for  each  of  the  seven  variables  considered.  Reporting  of  total  expenses  steadily 
improved  from  38  percent  in  1982  to  50  percent  in  1990.  Reporting  of  liability  premiums 
similarly  improved  from  38  percent  to  57  percent.  Large  improvements  were  also  found  in 
the  reporting  of  office  expenses,  non-physician  supplies,  payroll,  equipment,  and  other 
expenses.  Unfortunately,  however,  trends  have  been  somewhat  erratic.  Since  1988,  there 
have  been  declines  in  cumulative  response  rates  for  total  expenses,  liability  premiums,  and 
non-physician  supplies.  Reporting  on  office  expenses,  non-physician  supplies,  payroll,  other 
expenses  and  equipment  have  remained  relatively  stable.  These  findings  suggest  that 
problems  of  nonresponse  are  potentially  serious.  In  1992,  the  cumulative  response  rate  for 
each  of  the  seven  variables  considered  was  48  percent  or  less.  In  designing  future  practice 
cost  surveys,  researchers  should  be  cognizant  of  the  effect  of  both  item  nonresponse  and  unit 
nonresponse.  At  the  present  time,  it  should  be  acknowledged  that  cumulative  nonresponse 
rates  are  exceeding  the  levels  most  health  service  researchers  would  find  desirable. 

4.2.4  Measurement  Error 

Measurement  error  can  result  from  at  least  four  different  sources:  1)  the  respondent; 
2)  the  interviewer;  3)  the  questionnaire;  and  4)  the  mode  of  interview.  Measurement  error 
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Table  4-2:    The  Product  of  Survey  Response  and  Item  Response  Rates  for  Selected  Practice 
Cost  Questions 


Total 

Liability 

Office 

Year 

ExDenses 

Premium 

ExDenses 

1982 

0.38 

0.38 

0.34 

1983 

0.37 

0.43 

0.39 

1984 

0.40 

0.49 

0.43 

1985 

0.41 

0.48 

0.42 

1986 

0.41 

0.49 

0.42 

1987 

0.44 

0.51 

0.41 

1988 

0.47 

0.54 

0.44 

1989 

0.50 

0.56 

0.48 

1990 

0.50 

0.57 

0.51 

1991 

0.44 

0.52 

0.44 

1992 

0.41 

0.48 

0.43 

Non- 

physician 

Other 

SuDDlies 

Pavroll 

Expenses 

Equipment 

0.33 

0.33 

0.32 

0.38 

0.39 

0.36 

0.36 

0.42 

0.43 

0.41 

0.40 

0.41 

0.42 

0.40 

0.39 

0.41 

0.42 

0.40 

0.38 

0.39 

0.40 

0.39 

0.37 

0.42 

0.43 

0.41 

0.40 

0.46 

0.47 

0.45 

0.44 

0.49 

0.50 

0.48 

0.48 

0.41 

0.44 

0.40 

0.40 

0.41 

0.44 

0.41 

0.40 

Source:  AMA's  Socioeconomic  Monitoring  System  Core  Surveys. 
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occurs  when  respondents  give  inaccurate  answers  or  an  accurate  answer  is  incorrectly  coded. 
Measurement  bias  is  likely  to  be  a  particularly  serious  problem  in  surveys  of  physician 
practice  costs.  The  type  of  information  requested  in  these  surveys  is  unusually  detailed  and 
often  quite  burdensome  to  the  physician.  Although  some  evidence  presented  in  Chapter  2 
suggests  that  proxy  respondent-provided  information  may  be  better  than  information  from 
physician  respondents,  surveys  have  been  designed  primarily  to  obtain  information  directly 
from  physicians.  In  many  cases,  the  physician  must  refer  to  records  in  order  to  respond 
accurately.  In  other  cases,  the  physician  may  not  have  the  records  available  to  provide  the 
requested  information,  but  may  instead  offer  a  poorly  thought-out  estimate  in  a  misguided 
attempt  to  be  helpful.  Interviewers  rarely  are  trained  to  understand  the  implications  of  such 
"guesstimates."  In  many  cases,  it  is  preferable  to  impute  a  missing  value  rather  than  to 
encourage  a  physician  to  give  an  answer  that  may  be  inaccurate.  Because  obtaining  high 
response  rates  on  physician  surveys  is  so  difficult,  interviewers  may  also  be  giving 
inappropriate  positive  reinforcement  to  physicians  who  are  proceeding  through  the  survey 
without  break  off  even  though  they  do  not  appear  to  be  answering  questions  thoughtfully. 

Researchers  often  conduct  verification  surveys  or  record  checks  to  accurately 
determine  the  level  of  measurement  bias  and  make  appropriate  adjustments.  However,  such 
checks  are  not  possible  in  surveys  of  physicians.  This  is  why  the  survey  methods  literature 
on  physicians  surveys  is  so  sparse  compared  to  surveys  of  other  types  of  social  phenomena. 
In  the  vast  majority  of  cases,  we  have  no  "gold  standard"  to  which  survey  estimates  can  be 
compared. 

One  indirect  method  of  assessing  the  validity  of  answers  obtained  on  surveys  of 
physicians  is  to  examine  the  logical  consistencies  between  answers.  On  the  1988  NSOP, 
total  practice  costs  were  measured  in  two  ways.  First,  data  were  collected  about  the  specific 
components  of  practice  costs,  including  office  space,  purchase  of  equipment,  maintenance  of 
equipment,  and  wages  and  benefits.  The  individual  practice  cost  components  were  summed 
and  compared  to  the  response  to  a  separate  survey  question  that  asked  for  an  estimate  of  total 
practice  costs.  For  73  percent  of  the  cases  for  which  data  were  obtained  for  all  practice 
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components,  the  constructed  sum  of  the  measures  differed  by  more  than  10  percent  from  the 
total  practice  cost  measure  obtained  using  the  single  question.  There  did  not  appear  to  be 
any  pattern  of  overestimating  or  underestimating  specific  practice  cost  items  (Berk  et  al., 
1989).  Similar  findings  were  reported  in  an  analysis  of  a  pilot  survey  designed  to  measure 
practice  costs:  "The  most  serious  data  quality  problems  are  inconsistencies  between  total 
practice  expenses  and  the  constructed  sum  of  all  cost  items  and  between  total  nonphysician 
wages  and  the  constructed  sum  from  numbers  of  full-time  equivalent  employees  and  average 
wage  rates.  For  most  respondents,  inconsistencies  between  these  data  elements  range  from  a 
few  percent  to  over  50  percent"  (Brown.  1990,  p.  43). 

Mode  of  interview  is  another  potential  source  of  measurement  bias.  Although  most 
physician  surveys  are  designed  to  be  completed  directly  by  the  physician,  interviewers 
usually  are  allowed  to  accept  the  answers  from  a  proxy  if  the  physician  cannot  or  will  not 
respond  directly.    As  indicated  above,  the  designation  of  a  proxy  respondent  is  not  random. 
The  physician's  decision  to  be  interviewed  directly  is  largely  a  function  of  practice  size 
(Chapter  2)  and  perhaps  the  physician's  value  of  time  (Marder  and  Thran,  1989):  group  and 
more  highly  paid  physicians  are  more  likely  to  ask  proxy  respondents  to  complete  survey 
interviews  for  them.  This  is  a  potential  source  of  measurement  bias. 

4.3  Findings  and  Conclusions 

Although  physician  surveys  have  been  conducted  on  a  regular  basis  and  are  used  by 
the  federal  government  to  affect  payments,  there  is  little  solid  empirical  evidence  to  suggest 
that  these  data  are  accurate.  Because  there  are  no  pure  standards  which  can  be  used  to  assess 
the  potential  magnitude  of  bias,  no  definitive  conclusions  are  possible.  Nevertheless,  the 
high  levels  of  unit  nonresponse  coupled  with  high  levels  of  item  nonresponse  for  a  small  but 
important  set  of  practice  components  are  of  concern.  Moreover,  significant  logical 
inconsistencies  between  data  elements  have  been  reported.  Factors  such  as  the  use  of  proxies 
and  the  utilization  of  computer-assisted  telephone  interviewing  also  appear  to  affect  results 
(Project  HOPE,  1991). 
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To  the  extent  that  surveys  fail  to  measure  practice  costs  accurately,  payment  rates 
may  be  inequitable  both  for  the  physician  as  well  as  for  the  patient  who  pays  for  services. 
Results  from  recent  surveys  would  suggest  the  need  for  additional  methodological  inquiry. 
In  particular,  it  would  be  worthwhile  to  pursue  smaller-scale  studies  of  practice  costs  that 
would  allow  for  a  validation  analysis.  Obviously,  the  cost  of  such  inquiry  is  high,  but  in 
view  of  the  need  to  obtain  more  accurate  estimates,  such  investment  is  worthwhile.  There 
are  several  different  formats  that  could  be  explored. 

Physicians  who  initially  respond  to  a  physician  practice  cost  survey  could  be  re- 
interviewed  by  a  more  elite  group  of  survey  interviewers  who  would  ask  detailed  questions 
about  how  the  physician  was  answering  specific  questions.    Another  option  is  to  recruit  a 
sample  of  physicians  to  participate  in  focus  groups  designed  to  provide  more  information 
about  the  responses  given  during  the  survey.  It  also  may  be  useful  to  conduct  a  small-scale 
study  of  financial  records.  If  strict  confidentiality  can  be  guaranteed,  it  is  possible  that  a 
number  of  physicians  who  recognize  the  importance  of  these  data  in  calculating  payment 
rates  would  agree  to  participate. 

Finally,  we  would  recommend  that  policy  makers  consider  the  possibility  of 
developing  alternative  methods  of  collecting  data  that  rely  less  on  large,  national  survey 
efforts.  For  many  services  and  procedures,  the  work  component  of  the  relative  value  scale 
(which  underlies  the  new  Medicare  Fee  Schedule)  has  been  developed  using  expert  panels  in 
which  consensus  is  reached  about  the  amount  of  time  physicians  should  spend  performing 
specific  procedures.  A  similar  methodology  could  be  applied  to  estimate  practice  expense 
components.  It  could  be  argued  that  such  a  methodology  is  preferable  for  the  formulation  of 
public  policy.  The  goal  would  be  to  determine  appropriate  practice  costs,  rather  than  to 
estimate  average  costs.  Expert  panels  could  attempt  to  reach  consensus  on  the  appropriate 
equipment,  staffing,  and  other  resources  that  should  be  associated  with  specific  types  of 
localities  in  specifically  designated  markets.  While  such  a  methodology  is  subject  to  error,  it 
is  not  clear  that  it  would  be  any  less  accurate  than  the  far  more  burdensome  survey 
procedures  currently  used.  As  an  initial  step,  such  panels  could  be  conducted  for  three  to 
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four  specialties  and  the  results  compared  to  those  obtained  from  traditional  practice  surveys. 
This  may  inform  policy  makers  on  how  to  modify  existing  surveys,  or  whether  to  replace 
them  with  a  methodology  that  relies  less  on  the  ability  of  physicians  or  their  proxies  to  report 
detailed  information  about  costs. 
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APPENDIX: 


RATES  OF  REFUSAL  FOR  PHYSICIAN  AND  PROXY  RESPONDENTS, 

BY  SIZE  OF  PRACTICE 


Appendix:    Rates  of  Refusal  for  Physician  and  Proxy  Respondents, 
by  Size  of  Practice 


Physician  costs 


wages 


Physicians  <3  Physicians  >  3 

Physician  Proxy  Physician  Proxy 


7.2  7.9  8.0  5.5 


deferred  compensation  6.9  7.4  6.3  4.8 

fringe  benefits  6.8  7.4  6.8  4.4 


Office  employee  costs 

wages  3.4  3.2  2.1  1.6 

deferred  compensation  3.0  3.4  2.1  1.6 

fringe  benefits  3.2  3.4  2.1  1.8 

Medical  equipment 

depreciation  1.0  2.0  0.8  0.7 
Medical  supply 

expenses  1.1  1-8  0.8  0.9 


Source:    Analysis  of  1988  PPCIS  data  by  Project  HOPE. 

Note:  Differences  between  physician  and  proxy  rates  (for  each  practice  size  category)  are  not  statistically  significant 
at  an  alpha  level  of  0.0S. 


A-1 


CMS  LIBRARY 


flDT5  ooommb 


